ShaReD

Sharing and Reusing Data
(Video)

LLiB Colloquium

Jürgen Schneider

06 December 2023

What is video data in education?


shiksastudio on youtube

What is video data in education?

What is video data in education?


Diverse research designs

Depending on the research interest, e.g.


Connected with other data sources

Student assessment

Student survey

Teacher survey/interview

Why is video data powerful?

1. direct observation
grants insights into processes and practices of learning and instruction

2. high degree of richness/complexity

  • spoken word, intonation, modulation
  • facial expressions, gestures, postures
  • positioning and orientation in the room
  • temporal sequence and simultaneity of events

3. allows for nuanced evaluations
repeating or manipulating (vs. in situ observation)

(Derry et al., 2010; Lahn & Klette, 2023)

Why is video data powerful?

1. direct observation
grants insights into processes and practices of learning and instruction

2. high degree of richness/complexity

  • spoken word, intonation, modulation
  • facial expressions, gestures, postures
  • positioning and orientation in the room
  • temporal sequence and simultaneity of events

3. allows for nuanced evaluations
repeating or manipulating (vs. in situ observation)

(Derry et al., 2010; Lahn & Klette, 2023)

Major contributions

  • theory of teaching and learning
  • new perspectives in teaching research

TIMSS-Video (Stigler et al., 2000)
IPN-Video Study (Seidel et al., 2006)
Pythagoras-Study (Klieme et al., 2009)
PERLE-Study (Lipowsky et al., 2014)
DESI-Study (DESI-Konsortium, 2008)
IGEL-Study (Fauth et al., 2014)
TALIS-Video Study (Grünkorn et al., 2020)

Why is video data powerful?

1. direct observation
grants insights into processes and practices of learning and instruction

2. high degree of richness/complexity

  • spoken word, intonation, modulation
  • facial expressions, gestures, postures
  • positioning and orientation in the room
  • temporal sequence and simultaneity of events

3. allows for nuanced evaluations
repeating or manipulating (vs. in situ observation)

(Derry et al., 2010; Lahn & Klette, 2023)


video predestined for multiple analyses from different perspectives



high cost: producing quality video data heavily taxes financial, personal and time resources

ShaReD

ShaReD

Findable

ShaReD

Findable

ShaReD

Accessible

ShaReD

Interoperable

ShaReD

Reusable

Project phases & focus areas

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Possibilities & limits in the ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Possibilities & limits in the ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Possibilities & limits in the ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Project phases & focus areas

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Possibilities & limits in the ecosystem

  • Cooperation with similar projects

DEVELOPMENT

  • Measures for further development of the ecosystem

  • Expanding the availability of videos

  • Creation of guidelines, handouts, training courses

TESTING

  • User studies & evaluation

  • Implementation of first training courses with evaluation

  • Presentation of the concept at conferences

  • Development of dissemination strategies

SUSTAINABILITY

  • Materials for the implementation of training courses

  • Materials for the production of videos

Current projects

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Sharing:

  • 36 publications on the analysis of instructional videos in 2022

  • in 13 there is a “data statement” (all in journals with a corresponding commitment)


  • in 0 there is a “data statement” if this is not mandatory by the journal

  • classroom videos from 1 publication were shared (own website, not RDC)

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Sharing:

  • 36 publications on the analysis of instructional videos in 2022

  • in 13 there is a “data statement” (all in journals with a corresponding commitment)


  • in 0 there is a “data statement” if this is not mandatory by the journal

  • classroom videos from 1 publication were shared (own website, not RDC)

Reuse (FDZ Bildung):

  • 296 total applications (video data)
    (annual report 2022)
  • Coverage via reporting and google alerts underestimates coverage (covers approx. 75%) (Daniel et al., 2023)
  • Between 2016-2022: 8 publications and reports recorded (excluding students & teaching)

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Systematic Review
(top-down)

  • Search for
    • scientific studies
    • Position papers, guidelines, checklists, surveys
  • Focus on video data or qualitative data

Barriers and facilitators to sharing and reusing



Which factors contribute to classroom videos being shared or reused?



Systematic Review
(top-down)

  • Search for
    • scientific studies
    • Position papers, guidelines, checklists, surveys
  • Focus on video data or qualitative data

Delphi-Study
(bottom-up)

  • Experts:
    • FDZ employees
    • (Non-) reusers, (non-) providers
  • Complement and evaluate factors

Barriers and facilitators to sharing and reusing



Preregistration
https://doi.org/10.17605/OSF.IO/7FP5S


Open Data
So far on the github repo
(will be zenodo)

Barriers and facilitators to sharing and reusing

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Possibilities & limits in the ecosystem

  • Cooperation with similar projects

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?


Conceptual manuscript
(submitted)

  • Shows typical steps of sharing and reuse
  • Deliberately skips unspecific steps (e.g. data analysis)
  • Co-authors: Schreyer, Kunter, Porzelt

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?


Example: Informed Consent

  • classroom videos are necessarily personal data
  • need to specify the purpose, also those of future reuse
  • anticipation of reuse typically not possible

Guideline and resources for researchers



What are the steps involved in sharing/ reuse?
Which resources facilitate these steps?


Example: Informed Consent

  • classroom videos are necessarily personal data
  • need to specify the purpose, also those of future reuse
  • anticipation of reuse typically not possible

Resources

Guideline and resources for researchers

SCREENING

  • Which videos are available?

  • Which resources are available?

  • Possibilities & limits in the ecosystem

  • Cooperation with similar projects

Further aspects


Podcast episode


Episode on classroom videos


GEBF 2024

  • Contribution to symposium (Schreyer)
  • “Hilfreiche Ressourcen und Arbeitsschritte für die Bereitstellung oder Nachnutzung von Unterrichtsvideos”


Data acquisition

  • Extension of the offer to FDZ Education
  • Contact with several projects is slow



EARLI SIG 4/17 2024

  • Contribution “Collaborative space”
  • “Promoting Open Science in Learning Research: Collaborative Exploration of Future Directions”

Next steps

Next steps



  • Guideline on “Repository or research data center?”



  • pilot projects: Automated evaluation of classroom videos

References

Andersson, E., & Sørvik, G. O. (2013). Reality Lost? Re-Use of Qualitative Data in Classroom Video Studies. Forum Qualitative Sozialforschung, 14(3). https://doi.org/10.17169/fqs-14.3.1941
Corti, L. (2000). Progress and Problems of Preserving and Providing Access to Qualitative Data for Social ResearchThe International Picture of an Emerging Culture. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, Vol 1, No 3 (2000): Text . Archive . ReAnalysis. https://doi.org/10.17169/FQS-1.3.1019
Daniel, A., Jakowatz, S., Jung, N., Kleine, L., Kocaj, A., Meyermann, A., Mozygemba, K., & Schuster, A. (2023). Die Erfassung von Publikationen aus der Datennutzung: Verfahren, Herausforderungen und Nutzen.: Ein Erfahrungsbericht von Forschungsdatenzentren. RatSWD Working Paper. https://doi.org/10.17620/02671.80
Derry, S. J., Pea, R. D., Barron, B., Engle, R. A., Erickson, F., Goldman, R., Hall, R., Koschmann, T., Lemke, J. L., Sherin, M. G., & Sherin, B. L. (2010). Conducting Video Research in the Learning Sciences: Guidance on Selection, Analysis, Technology, and Ethics. Journal of the Learning Sciences, 19(1), 3–53. https://doi.org/10.1080/10508400903452884
DESI-Konsortium (Ed.). (2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESI-Studie. Beltz.
Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning and Instruction, 29, 1–9. https://doi.org/10.1016/j.learninstruc.2013.07.001
Grünkorn, J., Klieme, E., Praetorius, A.-K., & Schreyer, P. (2020). Mathematikunterricht im internationalen Vergleich. Ergebnisse aus der TALIS-Videostudie Deutschland. pedocs.
Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras Study. Investigating effects of teaching and learning in Swiss and German mathematics classrooms. [Online; Gedruckt]. In T. Janik & T. Seidel (Eds.), The power of video studies in investigating teaching and learning in the classroom (pp. 137–160). Waxmann.
Kristen, C., Stanat, P., Kogan, I., Lorenz, G., Gentrup, S., & Rahmann, S. (2018). Unterrichtsbeobachtung - KuL. Kompetenzerwerb und Lernvoraussetzungen (KuL). https://doi.org/10.7477/287:1:0
Lahn, L. C., & Klette, K. (2023). Reactivity beyond contamination. An integrative literature review of video studies in educational research. International Journal of Research & Method in Education, 46(1), 1–18. https://doi.org/10.1080/1743727X.2022.2094356
Lipowsky, F., Stubbe, T. C., Faust, G., Künsting, J., Hadeler, S., & Bos, W. (2014). Was leisten Schülerinnen und Schüler der privaten BIP-Kreativitätsgrundschulen im nationalen Vergleich? https://doi.org/10.25656/01:9678
Seidel, T., Prenzel, M., Rimmele, R., Dalehefte, I. M., Herweg, C., Kobarg, M., & Schwindt, K. (2006). Blicke auf den Physikunterricht. Ergebnisse der IPN Videostudie. https://doi.org/10.25656/01:4489
Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using Video Surveys to Compare Classrooms and Teaching Across Cultures: Examples and Lessons From the TIMSS Video Studies. Educational Psychologist, 35(2), 87–100. https://doi.org/10.1207/S15326985EP3502_3
Syring, M., Beck, N., Bohl, T., & Tesch, B. (2023). Klasse 6b. Eine Unterrichtswoche. Bildungswissenschaftliche und fachdidaktische Analysen. https://doi.org/10.15496/PUBLIKATION-75461
Troll, B., Pietsch, M., & Besser, M. (2023). Verhaltensbezogenes Engagement im Unterricht: Eine Analyse der Generalisierbarkeit und Zuverlässigkeit von Videobeobachtungen. Zeitschrift für Pädagogische Psychologie, 37(3), 157–171. https://doi.org/10.1024/1010-0652/a000286